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Abstract 


As  C2  systems  become  larger  and  more  complex,  architecture  plays  a  more  essential  role  during 
the  whole  life-cycle  of  the  systems.  Thereby,  capability  evaluation  of  C2  systems  at  architecture 
level  becomes  necessary  and  important  for  improving  the  system  capability  at  the  stage  of 
architecture  design.  This  paper  proposes  a  method  for  information  quality  evaluation  of  C2  system 
at  architecture  level.  First,  the  information  quality  model  is  proposed,  including  measures  of 
information  quality  and  methods  of  weights  assignment  and  synthesis  of  the  measures.  Second,  a 
framework  for  systematically  conducting  architecture-level  information  quality  evaluation  is 
provided.  Finally,  based  on  the  framework,  an  experiment  is  conducted  to  validate  the  proposed 
information  quality  evaluation  method.  Experiment  results  show  that  our  proposed  method  can 
effectively  evaluate  the  information  quality  based  on  architecture  models  of  C2  systems,  which 
can  help  to  identify  key  factors  impacting  information  quality  and  improve  the  system  capability 
at  the  stage  of  architecture  design  of  C2  system. 

Keywords:  Information  Quality,  Measures,  Architecture 

1  Introduction 

Control  and  Command  (C2)  systems  are  composed  of  hardware  and  software  subsystems  for 
gathering  information  from  various  sensors,  processing  and  displaying  information,  and 
commanding  and  controlling  various  weapons  to  attack  threatening  targets.  C2  systems  are  often 
large-scale,  complex,  real-time  and  software-intensive  systems.  It  has  long  been  recognized  that 
"architecture"  has  a  strong  influence  throughout  the  development  life  cycle  of  a  system  [1], 
especially  for  the  development  of  large-scale  and  complex  systems,  in  which  the  development 
process  often  involves  multiple  stakeholders,  various  and  complicated  activities,  and  multiple 
development  stages.  With  the  development  of  network  centric  warfare  (NCW),  C2  systems 
become  larger  and  more  complex,  architecture  of  C2  system  plays  a  more  essential  role  during  the 
whole  life-cycle  of  the  systems.  Therefore,  capability  evaluation  of  C2  systems  at  the  stage  of 
architecture  design  becomes  necessary  and  meaningful. 

In  the  environment  of  NCW,  information  quality/superiority  plays  a  very  important  role  for 
winning  the  war  [2].  Hence,  information  quality/superiority  is  usually  an  important  measure  of  C2 
system  capabilities  [3].  There  already  exist  many  literatures  to  evaluate  the  information 
quality/superiority  of  C4ISR  system  or  its  subsystems.  Qiu  [4],  Liu  [5]  and  Chen  [6]  proposed 
several  methods  for  evaluating  the  information  quality  or  information  superiority  of  C4ISR  system. 
Yang  [7],  Zhao  [8],  Zhao  [9]  and  Greg  [10]  respectively  provided  the  methods  to  evaluate  the 
information  quality/superiority  of  Data  Link  Communication  Subsystem,  Military  Communication 
Network,  Communication  Subsystem,  and  Tactical  Military  Networks  in  C4ISR  system.  Besides, 
Quan  [11]  provided  the  evaluation  index  of  information  superiority  and  discussed  several 
evaluation  methods.  Zhu  [12]  and  Zhang  [13]  evaluated  the  information  superiority  of  Near  Space 
Information  System  and  Command  Automation  System,  respectively.  Sangheun  [14]  analyzed  the 
information  exchange  capability  of  battlefield  networks.  Though  these  works  have  provided  many 
effective  methods  to  evaluate  information  quality/superiority,  there  still  lacks  a  method  that  can 
evaluate  the  information  quality  based  on  the  architecture  design  of  C2  systems.  Therefore,  in  this 
paper,  we  propose  a  method  for  information  quality  evaluation  of  C2  system  at  architecture  level. 
First,  we  propose  an  information  quality  model  that  qualitatively  establishes  the  relations  between 


architecture  design  and  measures  of  information  quality.  Second,  a  framework  is  provided  to 
conduct  the  architecture-level  information  quality  evaluation  of  C2  system.  Finally,  an  experiment 
is  conducted  to  validate  our  proposed  method.  Via  results  analysis  in  the  experiment,  it  shows  that 
our  method  can  help  designers  to  identify  key  factors  in  architecture  design  which  significantly 
impacts  the  information  quality  of  C2  system  and  improve  the  architecture  designs. 

The  rest  of  paper  is  organized  as  follow:  Section  2  describes  the  information  quality  model. 
Section  3  provides  a  framework  (named  as  DSEA)  to  systematically  evaluate  the  information 
quality  of  C2  systems  at  architecture  level.  In  Section  4,  based  on  the  DSEA  framework,  a  case 
study  used  to  validate  our  proposed  information  quality  model  is  presented.  Finally,  the  paper  is 
concluded  and  the  future  work  is  discussed  in  Section  5. 

2  Information  Quality  Model 

2.1  Measures  of  Information  Quality 

According  to  the  hierarchical  view  of  measure  of  effectiveness  proposed  in  [16],  we  provide  the 
measures  of  information  quality  as  shown  in  Fig.l.  At  the  level  of  measure  of  C2  effective 
(MOCE),  we  define  the  measure  Information  Quality.  At  the  level  of  measure  of  performance 
(MOP),  we  define  five  measures:  Completeness,  Correctness,  Currency,  Relevance  and  Shared 
Extent.  At  the  level  of  dimensional  parameter  (DP),  as  this  paper  focus  on  the  information  quality 
evaluation  at  architecture  level,  we  define  the  static  and  dynamic  parameter  of  architecture  of  C2 
systems.  Such  parameters  may  cause  remarkable  impact  on  system’s  MoP  or  MoCE  under  certain 
circumstances.  For  example,  ihQ  Average  Hops  of  Information  Transmission  between  systems  is  an 
important  parameter  when  conducting  time-critical  attacks.  In  Fig.  1,  it  is  worth  noting  that 
Relevance  and  Shared  Extent  can  be  measured  statically  while  Completeness,  Correctness  and 
Currency  need  to  be  dynamically  measured.  That  is,  the  dynamic  parameter  are  measured  by 
executing  architecture  (i.e.,  via  simulation).  These  parameters  are  usually  calculated  based  on  the 
collected  data  during  the  execution  of  architecture. 
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Figure  1.  Measures  of  Information  Quality 


We  use  /com?  dcor-,  dcur^  dReh  ^SE  to  represent  the  Completeness,  Correctness,  Currency,  Relevance 
and  Shared  Extent,  respectively. 


Completeness  is  the  percentage  of  detected  targets  to  the  entire  targets.  Completeness  is  a 
function  of  time,  which  is  defined  as: 

number  of  the  detected  targets  at  time  t 


number  of  the  entire  targets  at  time  t 


•  Correctness  is  the  degree  to  which  the  detected  targets  agree  with  ground  truth.  Like 
Completeness,  Correctness  is  also  a  function  of  time.  Assuming  there  are  N  detected  targets, 
and  each  target  has  M  information  items  (such  as  velocity  and  location),  using  Gi{t)  and  Pi{t) 
to  respectively  represent  the  actual  feature  and  the  detected  feature  of  rth  target  at  time  t,  Gi{t) 
and  Pi{t)  are  respectively  defined  as: 

GXt)  =  {gM  /e{l,2,-,^}  7G{l,2,-,m} 

=  /e  {1,2, •••,«}  j 

Therefore,  the  error  of y'th  information  item  of  dh  target  at  time  t  is  defined  as: 

ER,{t)=\p,{t)-g,{t)\ 


The  correctness  of  dh  target  at  time  t  can  be  defined  as: 

m  / 

4.(o=i-£5£i?,yM 

where,  SERy  is  the  standardization  of  ERy.  Then  average  correctness  at  time  t  is  defined  as: 

•  Currency  is  the  delay  from  the  time  at  which  a  target  is  detected  to  time  at  which  the  target  is 
taken  to  users.  Currency  is  defined  as: 


^cur  ^  ^  whcrc,  is  the  currency  of  dh  target. 


7=1 


•  Relevance  is  the  proportion  of  detected  targets  that  are  related  to  the  tasks  at  hand.  Assuming 
there  are  N  detected  targets  and  K  tasks.  Relevance  is  defined  as: 

1  if  the  dh  target  is  related  to  yth  task 
0  if  the  dh  target  is  unrelated  to  yth  task 

•  Shared  Extent  is  the  ratio  of  number  of  system/nodes  that  shared  the  detected  targets  to  the 
number  of  system/nodes  that  can  share  these  detected  targets.  Assuming  there  are  N  detected 
targets.  Shared  Extent  is  defined  as: 

_  number  of  systems/nodes  that  shared  the  dh  target 
number  of  systems/nodes  that  can  share  the  dh  target 


4  =  Z4 


/  N  where. 


'N-K 


where, 


/=i  j=\ 


2.2  Weights  Assignment  of  Measures 

n 

Assuming  W=(wj,  W2,  ...,  Wy,  ...,  is  the  weight  of  the  five  measures,  =  l,w^.  >0,n  =  5. 

7=1 

W],  W2,  W3,  W4  and  W5  represent  the  weights  of  Completeness,  Correctness,  Currency,  Relevance 
and  Shared  Extent,  respectively.  The  weights  of  the  five  measures  can  be  calculated  based  on  the 
methods  provided  in  Appendix  A. 


2.3  Synthesis  of  Measurers 


To  evaluate  information  quality,  it  is  needed  to  synthesize  the  five  measures  defined  in  Section  2.1, 
with  experts’  preferences  on  them  (i.e.,  the  weights  of  the  five  measures).  We  choose  the  marginal 
substitution  method  of  multi-attribute  decision  making  theory  [15]  to  synthesize  the  five  measures. 
In  addition,  the  indifference  curve  of  Cobb-Douglas  preferences  is  picked  for  the  marginal 
substitution  method.  The  utility  function  of  Cobb-Douglas  preferences  is  used  to  obtain  the 
synthetic  value  of  the  five  measures.  Formula  (1)  is  the  function  prototype  of  Cobb-Douglas 
preferences.  Three  indifference  curves  are  shown  in  Fig.  2.  X  and  Y  can  be  regarded  as  two 
measures,  a  and  b  are  the  user’s  preferences  of  measures.  U  is  the  synthetic  value.  Different 
points  in  an  identical  curve  are  equivalent.  That  is  why  it  is  named  as  indifference  curve.  From  Fig. 
2,  we  know  that  U  will  be  unchanged  by  increase  inXif  7  is  reduced.  The  arrow  in  figure 
indicates  the  increase  direction  of  U  .  The  values  on  curve  U  =  3  are  better  than  the  values  on 
curves  U  =  2  or  U  =  \.  The  parameter  a  and  b  reflect  the  user’s  preference  of  measures. 

U{XJ)  =  XX\a  >0,b>0  (1) 


Figure  2.  Examples  of  Indifference  curve  of  Cobb-Douglas  preferences 

Based  on  the  function  prototype  of  Cobb-Douglas  preferences  (i.e.  Formula  (1)),  the  five  measures 
can  be  synthesized  using  formula  (2).  It  is  worth  noting  that  the  effectiveness  type  of  measures 
and  the  cost  type  of  measures  need  to  be  respectively  standardized  using  formulas  (3)  and  (4) 
before  synthesizing  them. 

T  —  T  ^2  T  ^2  J  ^4  T  W5 

^  ~^Com  ^Cor  ^  Cur  ^Rel  ^  SE 

Xj 

r  =— ,x  =maxx  (3) 

Xj 

rj= — ,x:=minx^^  (4) 

Xj 

3  Architecture-level  Information  Quality  Evaluation  Framework 

To  systematically  evaluate  and  analyze  the  information  quality  of  C2  system  at  architecture  level, 
we  provide  a  framework,  including  architecture  Design,  Simulation,  Evaluation  and  Analysis 
(DSEA),  as  shown  in  Fig.  3.  First,  it  is  needed  to  design  and  describe  the  architecture  of  C2 


systems.  It  is  worth  noting  that  only  the  part  of  architecture  that  is  related  to  information  quality 
(IQ)  is  required.  Architecture  can  be  described  using  different  architecture  modeling  methods  or 
ADLs.  We  recommend  describing  the  architecture  using  standard  modeling  methods  or  languages, 
such  as  DoDAF  [20]  and  UML  [22].  Measures  of  Relevance  and  Shared  Extent  need  to  be  refined 
in  this  step.  Second,  with  the  described  architecture  of  C2  systems,  it  is  needed  to  select  or 
develop  a  simulation  platform  to  execute  the  described  architectures  under  specific  scenarios. 
Before  execution,  it  is  usually  required  to  transform  the  architecture  models  to  simulation  models 
which  can  be  executed  by  the  selected  or  developed  simulation  platform.  In  this  step,  there  is  a 
need  of  determining  the  data  that  need  to  be  collected  during  simulation  and  then  specify  the 
measures  of  Completeness,  Correctness  and  Currency,  based  on  the  simulation  platform.  Third, 
according  to  the  information  quality  model  proposed  in  Section  2,  information  quality  of  systems 
can  be  calculated  based  on  the  simulation  results.  Finally,  via  analyzing  the  evaluation  result,  we 
can  find  the  key  factors  impacting  information  quality  and/or  make  the  decision  of  architecture 
design. 
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Figure  3.  The  DSEA  Framework 


4  Experiment  and  Analysis 

To  validate  our  proposed  information  quality  model,  we  conducted  an  experiment  which  includes 
three  parts:  planning  and  design  of  the  experiment  (Section  4.1)  which  introduces  the  selected  C2 
systems  and  scenario,  experiment  execution  (Section  4.2)  which  is  further  divided  into 
architecture  description,  model  transformation  and  simulation  execution,  and  analysis  of 
experiment  result  (Section  4.3)  which  analyzes  and  discusses  the  collected  experiment  results. 

4.1  Planning  and  Design 

4.1.1  Experiment  Design 

In  this  experiment,  we  selected  two  C2  systems  with  different  architecture  designs  for  evaluating 
their  information  qualities.  More  specifically,  these  two  systems  respectively  use 
center-collaboration  mode  and  sensor-to-shooter-collaboration  (shorten  as  S2S-collaboration) 
mode  in  their  architecture  design.  In  this  experiment,  we  designed  a  scenario  of  heading  off 
missile.  In  center-collaboration  mode,  when  sensors  (such  as  radar)  detects  the  threatening  targets 
(e.g.  enemy  missile),  they  send  information  to  command  and  control  posts.  The  command  and 
control  posts  then  make  decision  and  assign  weapon  system  (such  as  interception  missile  site)  to 


intercept  the  enemy  missile(s),  as  shown  in  Fig.  4a.  Differing  from  center-collaboration  model, 
sensors  can  send  information  to  weapon  system  directly  in  sensor-to-shooter-collaboration  model. 
Based  on  the  information,  the  weapon’s  control  system  can  independently  determine  how  to 
intercept  the  enemy  missile,  as  shown  in  Fig.  4b. 


In  this  scenario,  there  are  one  enemy  missile,  three  sensors  (i.e.,  a  satellite,  a  long  rang  radar  and  a 
short  rang  radar),  a  command  and  control  post  and  an  interception  missile  site.  The  experiment 
was  conducted  in  two  conditions:  noninterference  and  with  electronic  interference.  In  the 
condition  of  electronic  interference,  we  added  an  interference  source. 
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a.  center-collaboration  model  b.  S2S-collaboration  mode 

Figure  4.  two  kinds  of  collaboration  mode  for  heading  off  enemy  missile 


4.1.2  Participants 

Two  kinds  of  participants  took  part  in  this  experiment.  The  first  kind  of  participant,  including  a 
PhD  student,  two  master  students  and  an  engineer,  finished  the  architecture  description.  The 
second  kind  of  participant,  including  three  business  experts,  gave  the  weight  of  the  five  measures 
defined  in  Section  2.1  and  confirmed  the  experiment  results,  which  will  be  described  in  detail  in 
Section  4.3.2.  It  is  worth  noting  that  the  simulation  is  a  constructive  simulation.  No  participants 
take  part  in  it. 


4.2  Execution 

4.2.1  Architecture  Description 

At  the  beginning  of  this  experiment,  architectures  of  the  two  systems  were  described.  First,  one 
PhD  student  and  two  master  students  studied  the  system  design  documents  within  in  a  week. 
Second  the  two  master  students  captured  the  architecture  element  in  the  design  documents  and 
described  them  as  DoDAF  models  using  IBM  RSA  modeling  tool  [21],  within  three  days.  The 
characteristics  of  the  architecture  description  are  listed  in  Table  1 . 


4.2.2  Model  Transformation  and  Simulation  Execution 


To  simulate  the  architecture  models,  we  built  a  simulation  platform  which  integrates  OPNET  [17] 


and  Simulink  [18]  via  MAK/RTI  [19]  in  this  experiment.  It  is  needed  to  transform  the  architecture 
model  to  simulation  models.  Based  on  the  defined  rules  in  our  previous  work  [23],  architecture 
models  conducted  in  Section  4.2.1  were  automatically  transformed  into  the  models  which  can  be 
directly  inputted  to  the  built  simulation  platform.  As  mentioned  in  Section  4.1.1,  the  whole 
simulation  was  divided  into  two  groups  (i.e.,  noninterference  and  interference).  In  each  groups,  30 
times  simulations  were  executed. 

Table  1 .  Characteristics  of  Architecture  Descriptions 


Models 

Descriptions 

Center-Collaboration  Mode 

S2S-Collaboration  Mode 

OV-2 

5  nodes  and  7  connections  among  them 

4  nodes  and  4  connections  among  them 

OV-4 

5  nodes  and  4  relationships  among  them 

4  nodes  and  3  relationships  among  them 

OV-5 

8  activities  and  23  flows  among  them 

7  activities  and  14  flows  among  them 

OV-6b 

20  states  and  27  transitions  among  them 

16  states  and  22  transitions  among  them 

OV-6c 

6  systems  and  14  events  among  them 

5  systems  and  15  events  among  them 

OV-7 

9  data 

7  data 

SV-1 

6  systems  and  their  16  interactions 

5  systems  and  their  12  interactions 

SV-2 

5  systems  and  their  4  communications 

4  systems  and  their  3  communications 

SV-4 

27  functions 

24  functions 

SV-5 

39  mappings  between  8  activities  and  27 

32  mappings  between  7  activities  and  24 

functions 

functions 

SV-1  Ob 

16  states  and  22  transitions  among  them 

14  states  and  19  transitions  among  them 

SV-lOc 

6  systems  and  29  events  among  them 

5  systems  and  19  events  among  them 

4.3  Results  and  Analysis 

4.3.1  Results  Collection 

After  each  simulation,  the  Icom,  Icor,  and  Icur  were  calculated.  In  this  experiment,  since  there  is 
only  one  enemy  missile,  the  Icom  was  calculated  by  averaging  the  detection  rate  of  the  three 
sensors  (i.e.,  the  satellite,  the  long  rang  radar  and  the  short  rang  radar).  For  Icor,  we  only  calculated 
one  information  item  “distance  precision  of  the  enemy  missile”  which  is  the  average  of  distance 
precisions  reported  by  the  three  sensors.  The  standardized  simulation  results  in  noninterference 
group  and  in  interference  group  are  respectively  listed  in  Table  6  and  Table  7,  Appendix  B 

4.3.2  Analysis  and  Discussion 

Using  the  average  results  in  Table  6  and  Table  7,  Appendix  B,  we  compare  the  Icom,  Icor,  and  Icur 
of  center-collaboration  mode  and  S2S-collaboration  mode,  as  shown  in  Fig.  5,  Fig.6  and  Fig.  7, 
respectively.  From  Fig.  5  to  Fig.7,  one  can  see  that,  in  the  condition  of  noninterference,  the 
completeness  {Icom)  and  correctness  {Icor)  of  these  two  kinds  of  mode  are  almost  the  same,  while 
for  the  timeliness  {Icur),  S2S-collaboration  mode  is  better  than  center-collaboration  mode  because 
sensors  can  directly  send  information  to  weapons  in  the  S2S-collaboration  mode.  In  the  condition 
of  interference,  from  Fig.  5  to  Fig.  7,  we  can  see  that  Icom,  Icor,  and  Icur  in  both 
center-collaboration  mode  and  S2S-collaboration  mode  become  worse.  Especially,  Icom  in 
S2S-collaboration  mode  is  reduced  much  more  than  which  in  center-collaboration,  as  shown  in 
Fig.  5. 


Figure  5.  Comparison  of  completeness  in  center-collaboration  and  self-collaboration 


Figure  6.  Comparison  of  correctness  in  center-collaboration  and  self-collaboration 


Figure  7.  Comparison  of  currency  in  center-collaboration  and  self-collaboration 


To  calculate  the  information  quality,  we  first  derived  the  weights  of  the  five  measures  from 
participated  experts,  as  listed  in  Table  2  (see  also  Appendix  A).  Second,  we  calculated  the  values 
of  relevance  {Irsi,)  and  shared  extent  {Ire,)  by  analyzing  the  architecture  design  of  the  two  systems 
(described  in  Section  4.1.1).  Finally,  the  information  qualities  were  calculated  based  on  the 
method  proposed  in  Section  2.3,  as  listed  in  Table  3. 

Table  2.  Weights  of  the  five  Measures 


Icom 

Icor 

^Cur 

hek 

ISE 

Weights 

9 

1 

8 

5 

3 

Standardized  Weights 

0.28 

0.22 

0.25 

0.16 

0.09 

Table  3.  Results  of  Information  Quality 


^Com 

Icor 

Icur 

hek 

ISE 

I 

Center-collaboration 

0.90 

0.84 

0.91 

1 

0.57 

0.87 

Noninterference 

S2S-collaboration 

0.91 

0.84 

0.94 

1 

0.6 

0.88 

Center-collaboration 

0.86 

0.66 

0.90 

1 

0.57 

0.81 

Interference 

S2S-collaboration 

0.78 

0.62 

0.94 

1 

0.6 

0.79 

From  the  column  ‘'Icur'  of  Table  3,  we  can  see  that  S2S-collaboration  mode  performs  better  than 
Center-collaboration  mode,  either  in  the  condition  of  noninterference  or  in  the  condition  of 
interference.  Via  analysis  of  the  architecture  design  described  in  Table  1  in  Section  4.1,  we  can  see 
the  fact  that  less  interactions  among  nodes/systems  in  the  S2S-collaboration  mode  save  the 
transmission  time  and  lead  to  high  currency.  For  example,  in  OV-5  and  SV-IOC,  activities/systems 
and  flows/events  among  them  in  the  S2S-collaboration  mode  are  obviously  fewer  than  those  in  the 
Center-collaboration  mode.  In  other  words,  target  information  are  transmitted  and  processed 
through  a  longer  path  and  more  systems  in  the  Center-collaboration  mode,  which  leads  to  the  low 
currency.  For  the  measures  completeness  {Icom)  and  correctness  {Icor),  by  analyzing  the  described 
architecture,  we  found  that  these  two  measures  are  mainly  influenced  by  the  ability  of  detecting 
and  identifying  targets.  According  to  the  architecture  design  presented  in  Section  4.1.1,  only  the 
C2  post  equips  the  multi-sensor  information  fusion  system.  In  the  Center-collaboration  mode,  all 
situation  awareness  information  gathered  by  different  sensors  are  converged  into  the  C2  post  and 
then  used  to  calculate  target  position.  Via  information  fusion  and  revision,  such  mode  can 
significantly  improve  the  completeness  and  correctness  in  the  condition  of  interference. 
Oppositely,  in  S2S-collaboration  mode,  the  weapon  system  highly  depends  on  a  single  sensor  (i.e. 
short  range  radar)  to  gain  the  target  position.  Hence,  the  interference  to  the  sensor  can  seriously 
reduce  the  information  quality  of  weapon  system  and  even  cause  it  to  lose  the  target. 

Evaluating  information  quality  of  C2  systems  at  architecture  level  aims  at  providing  suggestions 
for  improving  and/or  making  decision  of  the  architecture  design  of  C2  system.  Based  on  the  above 
analysis  and  the  results  listed  in  Table  3,  we  can  conclude  that  using  S2S-collaboration  mode 
could  provide  a  little  higher  information  quality  in  condition  of  noninterference  whereas  using 
center-collaboration  mode  could  provide  higher  information  quality  in  the  condition  of 
interference.  The  evaluation  conclusion  was  supported  by  the  participated  experts. 


5  Conclusion  and  Future  Work 


Control  and  Command  (C2)  systems  are  large-scale,  complex,  real-time  and  software-intensive 
systems.  As  C2  systems  become  larger  and  more  complex,  architecture  plays  a  more  essential  role 
during  the  whole  life-cycle  of  the  systems.  Information  quality  is  usually  an  important  measure  of 
C2  systems  effectiveness.  However,  in  existing  literatures,  there  is  a  lack  of  a  method  that  can 
evaluate  the  information  quality  of  C2  system  based  on  architecture  designs.  Therefore,  in  this 
paper,  we  propose  a  method  for  architecture-level  information  quality  evaluation,  including 
measures  of  information  quality  and  the  methods  for  weights  assignment  and  synthesis  of 
measures.  Moreover,  a  framework  is  provided  to  systematically  conducting  information  quality 
evaluation  of  C2  system  at  architecture  level.  With  our  proposed  information  quality  model,  one 
can  effectively  evaluate  the  information  quality  based  on  architecture  models  of  C2  systems.  In 
next  work,  we  intend  to  extend  the  experiment  by  launching  multiple  enemy  targets  and  apply  our 
proposed  method  to  more  cases. 


Appendix  A  Two  Kinds  of  Weight  Assignment  Model 


Experts  can  give  their  preferences  of  the  five  measures  via  two  methods:  1)  directly  giving  out  the 

X. 

importance  of  each  measure.  In  this  case,  Wj  can  be  gained  by  Wj  =  — — ,  where,  Xj  is  the 

7=1 

importance  of  the  jth  measure,  which  can  be  given  according  to  Table  4;  2)  giving  the  importance 
ratio  of  two  measures  as  matrix^.  The  Least  square  method  can  be  used  to  calculate  the  subjective 

n  n  n 

weight  by  minz  =  =l,w^.  >0  (The  detailed  description  is  presented 

i=\  j=\  j=\ 

in  section  3.1.2  of  Ref.  [15]).  Xy  is  the  importance  ratio  of  measure  i  to  measure  y,  which  can  be 
given  according  to  Table  5. 


Xii 

X12 

wjw^ 

1 - 

A  = 

X21 

^22 

^2n 

Wj/w, 

^n2 

1 

- 1 

Table  4.  Qualitative  concept  of  users’  preferences 


MI 

VI 

I 

LI 

NI 

LUI 

UI 

VUI 

MUI 

9 

8 

7 

6 

5 

4 

3 

2 

1 

M,  V,  I,  L,  N  and  U  are  respectively  used  for  Most,  Very,  Important,  Little,  Normal  and  Un 


Table  5.  Qualitative  ratio  of  users’  preferences 


G,  V,  I,  L,  S  and  U  are  respectively  used  for  Great,  Very,  Important,  Little,  Same  and  Un 

The  first  method  compares  importance  of  all  measures  and  assigns  an  importance  to  each  of  them. 
On  the  other  hand,  the  second  method  only  compares  measure  j  with  measure  j)  in  matrix  A. 
If  there  are  fewer  measures,  the  first  method  is  easier  for  experts  than  the  second  one.  But  second 
method  is  easier  than  the  first,  in  that,  experts  can  give  preferences  if  there  are  many  measures.  It 
is  indicated  by  psychology  experiments  that  ordinary  people  can  distinguish  five  to  nine  grades, 
and  researchers  suggest  using  nine  grades  and  selecting  the  integer  between  0  and  10  to  quantify 
the  qualitative  value  [15].  Thereby,  nine  grades  (i.e.  from  1  to  9)  are  used  in  Table  4  and  Table  5. 


Appendix  B  Standardized  Simulation  Result  in  Experiment 


Table  6.  Standardized  Simulation  Results  (noninterference) 


Center-Collaboration  Mode 

S2S-Collaboration  Mode 

iNO.  — 

Icom 

^Cor 

^Cur 

Icom 

Icor 

Icur 

1 

1.00 

0.83 

0.89 

1.00 

0.86 

0.93 

2 

1.00 

0.83 

0.90 

0.33 

0.98 

0.93 

3 

1.00 

0.88 

0.90 

1.00 

0.96 

0.93 

4 

1.00 

0.95 

0.91 

1.00 

0.87 

0.94 

5 

1.00 

0.94 

0.90 

1.00 

0.89 

0.94 

6 

1.00 

0.88 

0.89 

1.00 

0.66 

0.93 

7 

1.00 

0.81 

0.90 

1.00 

0.67 

0.93 

8 

1.00 

0.94 

0.90 

1.00 

0.84 

0.94 

9 

1.00 

0.81 

0.90 

1.00 

0.86 

0.96 

10 

1.00 

0.80 

0.90 

0.33 

0.98 

0.93 

11 

0.33 

0.50 

0.93 

1.00 

0.82 

0.96 

12 

0.67 

0.85 

0.93 

1.00 

0.87 

0.93 

13 

1.00 

0.88 

0.89 

1.00 

0.94 

0.94 

14 

1.00 

0.89 

0.91 

1.00 

0.91 

0.96 

15 

1.00 

0.82 

0.89 

1.00 

0.87 

0.94 

16 

1.00 

0.84 

0.90 

1.00 

0.90 

0.94 

17 

1.00 

0.82 

0.89 

1.00 

0.89 

0.94 

18 

1.00 

0.79 

0.89 

1.00 

0.91 

0.94 

19 

1.00 

0.93 

0.90 

0.33 

0.47 

0.93 

20 

0.67 

0.97 

0.93 

1.00 

0.90 

0.94 

21 

1.00 

0.81 

0.90 

1.00 

0.94 

0.94 

22 

0.33 

0.47 

0.93 

0.67 

1.00 

0.93 

23 

1.00 

0.85 

0.90 

1.00 

0.88 

0.94 

24 

1.00 

0.78 

0.89 

1.00 

0.79 

0.94 

25 

1.00 

0.81 

0.90 

1.00 

0.77 

0.96 

26 

0.67 

0.87 

0.93 

0.67 

1.00 

0.93 

27 

1.00 

0.79 

0.89 

1.00 

0.75 

0.96 

28 

1.00 

0.91 

0.92 

1.00 

0.84 

0.94 

29 

1.00 

0.91 

0.92 

1.00 

0.50 

0.94 

30 

0.33 

0.95 

0.90 

1.00 

0.73 

1.00 

Average 

0.90 

0.84 

0.91 

0.91 

0.84 

0.94 

Table  7.  Sta 


Center-Collaboration  Mode 
No.  - 


^Com 

Icor 

Icur 

1 

1.00 

1.00 

0.93 

2 

0.67 

0.60 

0.89 

3 

1.00 

0.62 

0.90 

4 

0.33 

0.68 

0.93 

5 

1.00 

0.62 

0.91 

6 

0.33 

0.64 

0.93 

7 

1.00 

0.63 

0.93 

8 

1.00 

0.67 

0.93 

9 

1.00 

0.61 

0.90 

10 

0.67 

0.61 

0.90 

11 

1.00 

0.62 

0.89 

12 

1.00 

0.57 

0.90 

13 

1.00 

0.69 

0.92 

14 

1.00 

0.69 

0.93 

15 

1.00 

0.64 

0.90 

16 

0.67 

0.60 

0.90 

17 

1.00 

0.73 

0.93 

18 

1.00 

0.60 

0.90 

19 

0.33 

0.62 

0.92 

20 

1.00 

0.63 

0.90 

21 

0.33 

0.94 

0.78 

22 

1.00 

0.61 

0.91 

23 

1.00 

0.58 

0.90 

24 

1.00 

0.80 

0.90 

25 

1.00 

0.65 

0.93 

26 

0.33 

0.68 

0.93 

27 

1.00 

0.66 

0.93 

28 

1.00 

0.62 

0.92 

29 

1.00 

0.69 

0.93 

30 

1.00 

0.65 

0.89 

Average 

0.86 

0.66 

0.91 

S2S-Collaboration  Mode 


^Com  ^Cor  ^Cur 


0.33  0.62  0.96 

1.00  0.46  0.78 

1.00  0.60  0.94 

0.67  0.69  0.93 

1.00  0.69  0.93 

0.67  0.69  0.93 

0.33  0.58  0.96 

0.33  0.58  0.96 

1.00  0.65  0.96 

1.00  0.70  0.93 

1.00  0.58  0.94 

1.00  0.65  0.93 

1.00  0.64  0.93 

0.33  0.57  0.94 

1.00  0.61  0.94 

1.00  0.69  0.93 

0.67  0.62  0.96 

1.00  0.68  0.96 

1.00  0.72  0.93 

1.00  0.59  0.94 

0.67  0.66  0.93 

1.00  0.64  0.93 

1.00  0.67  0.93 

1.00  0.56  0.96 

0.33  0.59  0.94 

0.33  0.47  0.93 

0.33  0.64  0.93 

1.00  0.55  0.96 

0.33  0.66  0.96 

1.00  0.63  0.96 

0.78  0.62  0.94 
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Introduction 


■  Information  quality/superiority  plays  a  very  important  role 
for  winning  the  war 

■  Information  quality/superiority  is  usually  an  important 
measure  of  C2  system  capabilities 

■  In  literatures,  there  lacks  a  method  that  can  evaluate  the 
information  quality  based  on  the  architecture  design  of 
C2  systems 

■  Therefore,  we  propose  a  method  to  evaluate  information 
quality  of  C2  system  at  architecture  level 


Information  Quality  Model 


Measure  of  C2  Effectiveness 
(MoCE) 


Information  Quality 


Measure  of  Performance 
(MoP) 


Dimensional  Parameter 


Completeness 
- - 


Correctness 


Currency 
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Relevance 
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\ 
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Shared  Extent 
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Miss  Distanee 
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No.  of  Systems 
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No.  of 
Information 


■  Completeness  ■  Relevance 

■  Correctness  ■  Shared  Extent 

■  Currency 


Information  Quality  Model 


U{XJ)  =  X‘^Y\a>0,b>0 


T  T  ^1  r  ^2  T  ^3  r  ^4  r  ^5 
^  “  ^Com  ^Cor  ^ Cur  ^Re/  ^ SE 


X. 

r.  =  —  ,x|  =  maxx 

x; 


,  Xj  =  min  Xj 


r.  = 
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Architecture-level  Information  Quality 
Evaluation  Framework 


Experiment  and  Analysis 


Long  rang  radar  Long  rang  radar 


Experiment  and  Analysis 


Models 

Descriptions 

Center-Collaboration  Mode 

S2S-Collaboration  Mode 

OV-2 

5  nodes  and  7  connections  among  them 

4  nodes  and  4  connections  among  them 

OV-4 

5  nodes  and  4  relationships  among  them 

4  nodes  and  3  relationships  among  them 

OV-5 

8  activities  and  23  flows  among  them 

7  activities  and  14  flows  among  them 

OV-6b 

20  states  and  27  transitions  among  them 

16  states  and  22  transitions  among  them 

OV-6c 

6  systems  and  14  events  among  them 

5  systems  and  1 5  events  among  them 

OV-7 

9  data 

7  data 

SV-1 

6  systems  and  their  1 6  interactions 

5  systems  and  their  12  interactions 

SV-2 

5  systems  and  their  4  communications 

4  systems  and  their  3  communications 

SV-4 

27  functions 

24  functions 

SV-5 

39  mappings  between  8  activities  and  27 
functions 

32  mappings  between  7  activities  and  24 
functions 

SV-1  Ob 

1 6  states  and  22  transitions  among  them 

14  states  and  19  transitions  among  them 

SV-lOc 

6  systems  and  29  events  among  them 

5  systems  and  1 9  events  among  them 
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■  center-collaboration 

■  S2S-collaboration 
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^Com 

^Cor 

^Cur 

^Rek 

hs 

I 

Noninterference 

Center-collaboration 

0.90 

0.84 

0.91 

1 

0.57 

0.87 

S2S -collaboration 

0.91 

0.84 

0.94 

1 

0.6 

0.88 

Interference 

Center-collaboration 

0.86 

0.66 

0.90 

1 

0.57 

0.81 

S2S -collaboration 

0.78 

0.62 

0.94 

1 

0.6 

0.79 

Conclusion  and  Future  Work 


■  Information  quality  is  usually  an  important  measure  of  C2 
systems  effectiveness 

■  With  our  proposed  information  quality  model,  one  can 
effectively  evaluate  the  information  quality  based  on 
architecture  models  of  C2  systems. 

■  In  next  work,  we  intend  to  extend  the  experiment  by 
launching  multiple  enemy  targets  and  apply  our 
proposed  method  to  more  cases 


